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ABSTRACT 

In  an  earlier  report  J4rJ  a  consistent  estimation  scheme 
was  given  for  Gaussian  Markov  random  field  models.  In  this 
report  we  consider  some  statistical  properties  of  the  resulting 
estimate.  Specifically,  we  derive  an  expression  for  the  asym¬ 
ptotic  mean  square  error  of  the  estimate  for  a  general  model 
and  compare  the  efficiency  of  this  estimate  with  the  popular 
coding  estimate  for  a  simple  first  order  isotropic  model. 
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1 .  Introduction 

Markov  random  field  (MRF)  models  are  of  much  interest  in 
image  analysis  and  processing.  For  instance,  they  have  been 
used  in  texture  analysis  [2,3]  and  image  restoration  [4-6], 

They  are  also  of  interest  in  the  analysis  of  field  data  [7] . 

MRF  models  characterize  the  special  nature  of  the  statistical 
dependency  of  intensity  levels  over  a  neighborhood  in  an  image. 

If  y(s)  denotes  the  intensity  level  at  location  s,  then  y(s) 
is  written  as  a  linear  combination  of  intensity  levels  {y(s+r), 
r€N>  (where  N  is  known  as  the  neighbor  set  of  dependence)  and 
additive  noise.  The  members  of  set  N  are  pairs  of  integers 
(k,£)  not  including  (0,0).  For  example,  the  first  order  MRF 
model  results  when  N=  ((0»1)» (1,0), (0,-1),  (-1,0)}  and  the  second 
order  MRF  model  results  when  N=  { (-1 , 0) , (1 , 0 ) , (0 , -1) , (0 , 1 ) , (-1 ,-l) , 
(-1 , 1) ,  (1 , -1) , (1 , 1) } .  Each  MRF  model  is  characterized  by  a  set  of 
linear  weights  and  the  variance  of  the  additive  noise. 

Suppose  we  are  given  an  array  of  intensity  level  variations 
{y(s),sfSl},  fi={s:  (i,  j)  ,  lSi,jSM}  and  we  are  interested  in  fitting 
a  Gaussian  MRF  model  to  this  data.  We  also  assume  that  the  spec¬ 
ific  structure  of  the  MRF  model  characterized  by  the  neighbor  set 
N  is  known.  The  problem  of  estimating  N  has  been  considered  else¬ 
where  [1,8].  Three  estimation  methods,  namely  the  coding  method 
[7],  the  maximum  likelihood  (ML)  method,  and  the  method  in  [9] 
can  be  used  to  obtain  estimates  of  parameters  characterizing  a 
MRF  model.  It  is  the  intent  of  this  paper  to  analyze  in  some 
detail  the  statistical  properties  of  the  estimate  in  [1,8]. 


2.  Estimates  and  their  statistical  properties  in  MRF  models 
2.1  Model  Representation 


Assume  that  the  given  image  data  (y(s)}  obeys  the  MRF  model 
in  (2.1),  with  the  associated  neighbor  set  N: 


y(s)=  E  6  v(s+r)+e(s) ,s€fi,  (2.1) 

r€N  r 

In  (2.1)  the  neighbor  set  N  is  symmetric,  i.e.,  if  r€N  then 
-r€N.  Further,  the  coefficients  satisfy  the  constraint  6r=0_r. 
The  stationary  noise  sequence  (e(s)}  is  partly  characterized  by 


E(e(s) |y (r) )=0  Vs*r 


(2.2) 


Using  (2.1)  and  (2.2)  one  can  prove  that  the  noise  sequence 
(e(s)}  is  correlated  with  the  correlation  structure 


E(e(s)  e(r))=y,  s=r 

=-0s_rv, (s-r) €N  (2.3) 

=  0  otherwise 

It  can  be  shown  by  methods  similar  to  [10,11]  that  an  observation 
y (s)  obeying  (2.1)  with  the  conditions  (2.3)  indeed  satisfies 
the  Markov  condition 

P(y  (s)  |  y(r)  ,r€fl,rj<s) 

=  p(y  (s)|  y (s+t)  ,t*N) 

We  shall  characterize  the  neighbor  set  for  the  MRF  model  using 
the  set  Ng,  which  includes  members  from  the  non-symmetric  half 
of  N,  i.e.,  if  sfNg  then  -s€Ng  and  N  =  (s:s€Ng}u{-s ss€Ng} .  A 
sufficient  condition  to  ensure  stationarity  is  [7] 

^^k,^2!  2 2  K  1  w^enever  |zjJ  *  ]  z2  j  *  1 


2 . 2  A  consistent  estimation  scheme  [1>  8]  : 


Consider  the  estimates 


0*  =  [E  q(s)  qT(s)]-1(E  q(s)  y  (s)  )  (2.4) 

£2  T~  ~  ' 

and 

v*  -  ——  E  (y (s) -  9  *Tq ( s )  ) 2  (2.5) 

M2  £2  x  ~ 


where 


fij  =  !1  -  S2B 

and 


£2  =  {s  =  (i,j):  s€£2  and  (s+r)/£2 

JD 

for  at  least  one  r€N} 

We  now  state  a  theorem  regarding  the  consistency  of  the  estimate 

9*  and  give  an  expression  for  the  asymptotic  variance  of  the 

★ 

estimate  0  .  An  expression  for  the  asymptotic  variance  of  0*  for 
an  isotropic  MRF  model  with  Ng  =  {  (0 ,1) ,  (1 , 0) }  is  also  given. 


Theorem  1 ;  Let  y(s),  s€  (2  be  the  set  of  observations  obeying 
the  MRF  model  (2.1).  Then 

(i)  The  estimate  0*  is  asymptotically  consistent. 

(ii)  The  asymptotic  covariance  matrix  of  0*  is 
E(9-0*)  (0-0*) T=  -Jv  Q-1  +  2v2(QTQ)-1 

m2  ~  ~ 

(2.6) 


■7  9 

n 


-1  L  E 


s  r 

(s-r)€N 


where 


(iii)  For  the  isotropic  conditional  model  with  Ng  = 

{ (0,1) , (1,0) } ,  the  asymptotic  expected  mean  square  error  is 


,  262 (l-4ea.  n)2 

E(9-e*r  =  - 5—5 - ±1^— 

4M  «I,0 


(2.7) 


‘1,0 


=  cov (y (s) ,y (s+(l,0) ) ) 
cov(y  (s)) 


(2.8) 


The  elements  of  matrices  Q  and  T  _  are  functions  of  normalized 

~  ~r ,  s 

autocorrelation  coefficients  The  proof  is  given  in  Appen¬ 

dix  I. 

Although  0*  is  a  consistent  estimate  of  0,  it  is  not  very 
efficient.  We  compare  the  efficiency  of  this  estimate  with 
efficiencies  of  the  coding  estimate  for  a  simple  MRF  model  with 
Ng  =  { (0,1), (1,0) } .  The  exact  ML  estimate  is  obtained  by  assuming 
a  toroidal  lattice  representation  for  y(s)  and  maximizing  the 
resulting  log  likelihood  function  by  using  the  Newton-Raphson 
procedure.  Equation  (2.4)  can  also  be  used  for  toroidal  lattice 
representation  by  summing  over  (2  instead  of  The  resulting 

difference  in  the  numerical  value  is  negligible  for  sufficiently 
large  M. 


1 


2 . 3  Comparison  of  estimates 

We  compare  the  asymptotic  variance  of  the  estimate  (2.4) 
with  the  asymptotic  variances  of  the  coding  estimate  and  ML 
estimate  for  the  isotropic  conditional  models  with  Ng  =  {  (0,1), ( 
From  [9] ,  the  asymptotic  variance  of  the  coding  estimate  Qq  is 


-  0(1-400.  ft) 

M  Var<ec)  "  ~  2a  ~ — 


1,0 


(2.9) 


Also  from  [9] ,  the  variance  of  the  ML  estimate  9,„  is 

MIj 


Var(0ML)  =  3 


0.5 


(2.10) 


M  (1(0)  -  4V£0 (0) ) 


where 


IO) 


1  jj  (cos  x+cos  y)2dxdy 
4it2  0  (1-20  (cos  x  +cos  y)  ) 


and 


vst(9) 


2  7T 

1  (•  r  cos  ( sx+ty )  dxdy _ 

4ir2  J0J  (1-20  (cos  x  +  cos  y) ) 


Tabulated  values  of  V^q(0),  q  and  1(0)  are  available  in  [9] 
for  different  values  of  0.  Using  these  values,  and  (2.7),  (2.9 

(2.10),  Columns  2-4  of  Table  I  are  computed.  The  asymptotic 
efficiencies  in  columns  5  and  6  are  defined  by 


eff (0C) 


Var<6  ML> 
Var(0c) 


and 


eff  (0*) 


Var<§  ML> 

Var(9*) 


1,0)} . 


and 


It  is  evident  that  the  estimate  0*  computed  using  (2.4)  is 
more  efficient  than  the  coding  estimate  but  is  not  as  good 
as  the  ML  estimate.  Note  that  column  5  is  available  in  [9] . 


Appendix  I 


proof  of  Theorem  1_: 
i)  We  have  from  (2.4) 


€>*  =  C£q<s>qT(s>3_1(  ^q(s)y(s)) 


(1  ) 


Substituting  for  y(s)  from  (2-1)  and  simplifying,  we  have 


[^q(s)qT(s)](e*-e)  =  £q(s)e(s) 


(2) 


Since  E(q(s)e(s))  =  0  by  (2.2)  and  C  ]Pq(s)qT(s)]  is  a  positive  definite 


matrix,  the  consistency  of  the  estimate  e  follows. 


C*D 


ii)  To  make  our  calculations  easy  we  assume  from  now  on  thate(s)is  normally 
distributed.  Multiplying  (2)  by  its  transpose  and  taking  expectations, 
we  have 


ECj3q(s)q^(s)( e*-e) ( 6*-6  f(^q(s)q,(s))l3 
s'  ~  ~  s' 


_T,_^T, 


=  EC  53  q(s)e(s)  53  (q(r)e(r))^3 


(3) 


=  ££E<qCs)e(s))  E(qT(r)e(r)) 
s  r 

+5353E(g<s)e(r)>  E(gT(r)e(s)) 
s  r 


(4) 


♦£XE(e<s>e<r))  ECg(r)9T(s)> 


s  r 


s  i  +  II  +  III,  where 


I  =  0  , 


by  using  (2.2)^ 


II  =  2  M2v2  Imxm  ,  by  using  (2.2); 


(5) 


(6) 


(7) 


UI  =  v  £  E(q(s)qT(s)> 

s 


! 


for  large  values  of  M 


^V)q(s)qT(s)  =  Q  +  nCM), 
Ms' 


where  n(M)  is  such  that 


E(n2(M))  =  0 

Using  (11)  we  obtain 
(LHS  of  (3))/M4  = 

ECCQ  +  n(M) ) (e*-e) (6*-6)T(q  +  n(M))T] 


=>  Q  E(e*-0)(8*-6)T  Qr  +  0(1/M2), 


Substitution  of  (10)  and  (13)  into  (3)  yields 


E(e*-exe*-e)T  =  ^  Cv  q'1  +  2v2(q2)-1 

'  '  '  ' 


-  3  i'  EE  •(w,  Iris'S1"1  *  "(I/*  ^ 

M  s  r 

(s-r)cN 


(11) 


(12) 


(13) 


(14) 


Q.E.D. 

c)  For  the  isotropic  conditional  model  with  =  <(0/1)/  (1/0)}/  (14) 


reduces  to 


<15> 


♦  vE(q2(s>>  -  «v  ££  E(q(s)q(r) )] 

s  r 

(s-r) cN 


where 


q(s)  =  y(s+(0,1))  +  y(s+(0,-1)) 


+  y(s+(1,0>)  +  y(s+(-1,0)) 


(16) 


Let 


L  =  ECy(s)y(s+(k,l))3 


(17) 


Note 


Yk,l  =  Y-k  rLand  Yi,k  =  vk,i 


(18) 


Express  the  higher  order  correlations  y  t-j  -2'  Y2  0  an<*  T3  0 


terms  of  yQ,Q,  yQ,y  y^Q  and  by 


Y0,0  =  v/<1_4ea1/0) 


(19) 


Y2,1  =  h  Y1,1  “  Y1,C 


(20) 


5/0  =  Yl,0*1  +0|}  “  Y0,0  '  3y1,1  "  **1,-1 


20 


26 


(21) 


Y0,0  "  Y1,1 


(22) 


y 


2/0 


1 

•  Y1,0 


r1/-1 


yi,-2  =  ?e  Yi,-i  ~  Yi,o  <23) 

Equations  (19-23)  can  be  obtained  by  multiplying  y(s)  by  appropriately 
shifted  y(s+(k,l))  and  taking  expectations. 

Consider  the  various  terms  in  (15). 


E(,2(S»  =  I  *T1/0 


(24) 


'EY;  E(q(s)q(r))  =  4C9t1#(J  +  y%Q  +  3y2J  +  3Y,f.2l 
(s-r)eN 

—  AC4Yi^q  +  2  ^  Q  *0*  Tg  g] 

8 

Substitution  of  (24)  and  (25)  in  (15)  yields 


E(e*-e)z  =  1-  — —  C4vz 
Z 


M  .16y1/0 


which  on  using  (19)  and  q  = 


*  2  2e2(1“4ea1  q)2- 

E(e  -e)z  - -  -f-" — 


♦  ""0,0  - 

T1,0/Y1,1  9ives 


(25) 


Table  1.  Computation  of  Asymptotic  Variances  and  Efficiencies 
of  Different  Estimates  in  Isotropic  Conditional 
Model  with  N„  =  { (0, 1) , (1.0) } 
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